Sr Data Engineer | Full Time
PB Consulting
 New York, NY
Posted On: Jun 04, 2024 
Posted On: Jun 04, 2024 
Job Overview
Salary
Depends on Experience
Required Skills
- Scala
 - Python
 - Apache Spark
 - PySpark
 - AWS
 - SQL
 
Job Description
Roles and Responsibilities
- Develop products and services on the latest technologies through contributions in development, enhancements, testing, and implementation. 
 
- Work on migrating applications from an on-premises location to the cloud service providers. 
 
- Develop, modify, and extend code for building cloud infrastructure, and automate using CI/CD pipeline. 
 
- Partners with business and peers in the pursuit of solutions that achieve business goals through an agile software development methodology.
 
- Work with peers across the system to define and implement best practices and standards. 
 
- Assess applications and help determine the appropriate application infrastructure patterns. 
 
 
Required Experience/Skills
- Hands-on experience in building/implementing cloud platforms/applications on the AWS platform. 
 
- Experience in developing data pipeline solutions to ingest and exploit new and existing data sources. 
 
- Expertise in leveraging SQL, programming languages like Python, and ETL tools like Databricks.
 
- Expertise in AWS Compute (EC2, EMR), AWS Storage (S3, EBS), AWS Databases (RDS, DynamoDB), AWS Data Integration (Glue). 
 
- Good understanding of AWS Identify and Access management, AWS Networking, and AWS Monitoring tools. 
 
- Proficiency in CI/CD and deployment automation using the GITLAB pipeline. 
 
- Proficiency in one or more programming languages e.g., Python, Scala. 
 
Job ID: PC240203